Non-Random Sampling and Association Tests on Realized Returns and Risk Proxies

This paper investigates how data requirements can induce a non-random selection of observations from the reference sample to which the researcher wishes to generalize test results. We illustrate the effects of non-random sampling on results of association tests in a setting with data on one variable of interest for all observations, and frequently-missing data on another variable of interest. We develop and validate a bootstrapping approach to construct samples that approximate randomly-drawn samples, using only observations from the data-restricted subsample. Simulation tests show these distribution-matched samples yield generalizable results. We also demonstrate the effects of non-random sampling using archival tests of the association between realized returns and five implied cost of equity metrics. Here, the reference sample has full information on realized returns, but only 16% of reference sample observations have data on cost of equity metrics. In contrast to inferences from the unadjusted (non-random) cost of equity sample which shows weak or negative associations, distribution-matched samples show reliable evidence of the theoretically predicted positive association between realized returns and cost of equity metrics.